fix(install): stop leaking temp files when an atomic write fails - #165
fix(install): stop leaking temp files when an atomic write fails#165rominf wants to merge 1 commit into
Conversation
8c1ec0c to
7204154
Compare
Merge-queue ejection: root cause and fixThis PR was ejected from the merge queue when the blocking Root causeThe captured screen in that log shows the dashboard still on Home — the The two scenarios that open Observe launch the TUI and send the tab key with no assertion in between, unlike the demo-data journeys which assert the home view first. A key written into the pseudo-terminal before the dashboard is reading input can be consumed by whatever holds the terminal at that moment, and nothing retries it. The step then fails much later, in an assertion about a view the dashboard never left — which is exactly the failure signature above. This is a latent synchronisation defect in the E2E step, not in this PR's production code: nothing in the download/atomic-write change is reachable from FixSend the tab key repeatedly until the Observe chip is actually marked active, so the step depends on the dashboard having acted on the key rather than on it having been ready when the key was written. The helper is restricted to idempotent keys (a tab jump, not a toggle). ReproductionDeterministic, by pointing the harness at a wrapper that drains the terminal before exec'ing the real binary — i.e. a dashboard that is not yet reading when the key arrives: cat > /tmp/wrapper/rocm <<'SH'
#!/bin/sh
stty raw -echo min 0 time 20; dd bs=1 count=16 of=/dev/null 2>/dev/null; stty sane
exec /path/to/target/release/rocm "$@"
SH
ROCM_CLI_BINARY=/tmp/wrapper/rocm cargo xtask e2e -- -n "Observe displays metrics"Before: Verified in addition: full Note: the guard is exercised by these two scenarios on every run, but the key-loss condition itself is only reproducible with the wrapper above, so there is no automated test that would fail without it. |
7204154 to
78851e6
Compare
write_file_atomically named its scratch file path.tmp-<unix_ms> and returned early on a write or rename error without removing it. Because the name embeds a timestamp, each retry left a distinct orphan rather than reusing one — so on a full disk, retrying made things worse: the first orphan held all the space that remained. Remove the temp file on every failure path in both copies of the helper, and build the name from the whole file name so a multi-extension artifact keeps its extensions: sdk.tar.gz now yields sdk.tar.gz.tmp-<id>, where with_extension dropped the .gz. Refs #158 Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
78851e6 to
a1a7ad1
Compare
Re-scoped — please re-review, the existing approval predates thisHeads up that this PR now does substantially less than when it was approved. The Dropped: the download half. #198 (c6b5bb5) landed Kept: the Split out: the e2e TUI flake fix. The Observe-tab keystroke retry was
|
Dismissing automatically: this PR has been re-scoped since this approval. The download rewrite that was reviewed here has been dropped, because #198 (c6b5bb5) landed a more complete implementation of the same idea on main. What remains is a different change — the write_file_atomically temp leak in both copies of the helper — plus a corrected Refs #158 (it does not close #158; a .part file orphaned by a hard crash still survives, since the name is keyed on the pid). Re-review requested. See the re-scope comment for detail.
Summary
write_file_atomicallyleaves its scratch file behind when a write or renamefails. The name embeds a millisecond timestamp, so each retry leaks a distinct
orphan instead of reusing one — on a full disk, retrying makes things worse.
(
apps/rocm/src/therock.rs,apps/rocmd/src/lib.rs).keeps its extensions:
sdk.tar.gzyieldssdk.tar.gz.tmp-<id>wherewith_extensiondropped the.gz.Root cause
Reproduced by filling a small filesystem and calling the helper twice:
Two attempts, two orphans, the first holding all the space that was left.
Scope
This PR originally also rewrote the download path. #198 has since landed
download_file_streaming, which covers that ground more thoroughly (sibling.partfile, resume, length and digest cross-checks), so those changes aredropped in favour of it. What remains is the one defect #198 did not touch.
Note this does not close #158. Both in-call cleanup gaps are now fixed, but
a
.partfile orphaned by a hard crash or Ctrl-C still survives: the name iskeyed on the process id, so a later run never removes it, and nothing sweeps
for them. Leaving #158 open for that.
Test plan
write_file_atomically_cleans_up_temp_when_the_rename_fails(both crates) —portable, runs in CI. Fails before, passes after.
write_file_atomically_temp_name_preserves_multi_dot_file_names,temp_sibling_path_preserves_multi_dot_file_names.write_file_atomically_cleans_up_temp_on_write_failure(both crates) —the original
/dev/shmENOSPC reproduction,#[ignore]d because it fills ashared tmpfs. Run with
cargo test -- --ignored.Risk: low. Two self-contained helpers; behaviour on the success path is
unchanged apart from the temp file's name.
Refs #158